@cmpadden Thanks for your comment, i will undo this and raise a PR for your review shortly in 2 days. Thanks!

@cmpadden Thanks for your comment, i will undo this and raise a PR for your review shortly in 2 days. Thanks!

@cmpadden raised a new version. can you please help me get review for this. Thanks!

Thanks @cmpadden. The dropped TODO placeholders were restored across the docs in an earlier push, and I just fixed two remaining issues from your review:

• dagster-omni.rst: the restored TODO had been re-added without a newline, which folded the whole line (including the link sentence) into an RST comment. Since the state-backed components guide now exists, I replaced the TODO with the actual visible link.
• dagstermill/examples/repository.py: restored the # FIXME add an output to this marker that had been dropped.

Two honest notes: this PR still has merge conflicts with master, and it mixes docs cleanups with some larger core-logic changes (asset_backfill pagination, run_monitoring batching, child-process stdout/stderr capture). If it's easier to review, I'm happy to split the docs-only changes into a separate, smaller PR and pull the core-logic changes out into their own focused PRs. Let me know what you'd prefer.

Greptile Summary

Despite being titled "Docs: fill in TODOs and improve cross-links", this PR also lands a significant batch of Python backend improvements that resolve long-standing TODOs: streaming pagination for asset-key merging, child-process stdout/stderr capture, bulk dynamic-partition operations, SparkMemory/SparkTime validated config types, and a rewrite of the CLI-log event parser. Two of those backend changes introduce correctness problems that need attention before merge.

• filter_dagster_events_from_cli_logs rewrite (events/utils.py): the new raw_decode-based loop accumulates subsequent log lines onto a failed-parse buffer, so a single non-JSON {…} fragment in log output can silently prevent all later DagsterEvent objects from being extracted.
• BigQuery table_definitions (bigquery/configs.py + ops.py): the config schema uses snake_case keys (source_uris, source_format) but ExternalConfig.from_api_repr expects camelCase API keys (sourceUris, sourceFormat); user-supplied external-table definitions are silently discarded.
• All documentation, test-cleanup, and comment-tidying changes look correct and unambiguous.

Confidence Score: 3/5

Two backend changes introduce real correctness problems: the CLI-log event parser can silently lose DagsterEvents, and the new BigQuery table_definitions config is effectively non-functional due to a key-format mismatch. Everything else in the PR is safe.

The event-parser rewrite in utils.py can silently drop DagsterEvents whenever a log line contains a stray { before a valid event JSON, which is a behavioral regression affecting child-process execution visibility. The BigQuery table_definitions feature, while net-new, ships broken — users who configure source_uris will silently get an empty ExternalConfig. Both issues affect current changed code paths. The rest of the PR (docs, pagination refactors, Spark types, dynamic-partition bulk ops) is straightforward and correct.

python_modules/dagster/dagster/_core/events/utils.py (event-parser buffer logic) and python_modules/libraries/dagster-gcp/dagster_gcp/bigquery/ops.py + configs.py (table_definitions key format) need fixes before merge.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[get_assets GraphQL resolver] --> B{asset_keys provided?}
    B -- No --> C[_iter_asset_graph_keys\nsorted cached property]
    B -- No --> D[_iter_materialized_keys\npaginated DB fetch]
    C --> E[heapq.merge\nsorted by key_str]
    D --> E
    E --> F{key_str == last_key?}
    F -- Yes, duplicate --> G[skip]
    F -- No --> H[append to merged_asset_keys]
    H --> I{len >= limit?}
    I -- Yes --> J[return GrapheneAssetConnection]
    I -- No --> E
    B -- Yes --> K[use provided asset_keys directly]
    K --> J

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[get_assets GraphQL resolver] --> B{asset_keys provided?}
    B -- No --> C[_iter_asset_graph_keys\nsorted cached property]
    B -- No --> D[_iter_materialized_keys\npaginated DB fetch]
    C --> E[heapq.merge\nsorted by key_str]
    D --> E
    E --> F{key_str == last_key?}
    F -- Yes, duplicate --> G[skip]
    F -- No --> H[append to merged_asset_keys]
    H --> I{len >= limit?}
    I -- Yes --> J[return GrapheneAssetConnection]
    I -- No --> E
    B -- Yes --> K[use provided asset_keys directly]
    K --> J

_{Reviews (1): Last reviewed commit: "docs: fix omni state-backed link; restor..." | Re-trigger Greptile}

Closing in favor of #33940.

Per the earlier discussion about splitting this, I pulled the conflict-free docs improvements out into #33940 — a small, focused docs-only PR against current master. That's the part that was ready and reviewable. The remaining core-logic changes (asset_backfill pagination, run_monitoring batching, child-process stdout/stderr handling, etc.) had merge conflicts and some flagged correctness concerns, so rather than carry them in a sprawling 58-file PR I'll raise any that are still worth doing as separate, focused PRs. Thanks @cmpadden for the review!